Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes
نویسندگان
چکیده
Most exact algorithms for general par tially observable Markov decision processes (POMDPs) use a form of dynamic program ming in which a piecewise-linear and con vex representation of one value function is transformed into another. We examine vari ations of the "incremental pruning" method for solving this problem and compare them to earlier algorithms from theoretical and em pirical perspectives. We find that incremen tal pruning is presently the most efficient ex act method for solving POMDPs.
منابع مشابه
Incremental Pruning : A Simple , Fast , Exact Method for PartiallyObservable Markov Decision
Most exact algorithms for general partially observable Markov decision processes (pomdps) use a form of dynamic programming in which a piecewise-linear and convex representation of one value function is transformed into another. We examine variations of the \incremental pruning" method for solving this problem and compare them to earlier algorithms from theoretical and empirical perspectives. W...
متن کاملA POMDP Framework to Find Optimal Inspection and Maintenance Policies via Availability and Profit Maximization for Manufacturing Systems
Maintenance can be the factor of either increasing or decreasing system's availability, so it is valuable work to evaluate a maintenance policy from cost and availability point of view, simultaneously and according to decision maker's priorities. This study proposes a Partially Observable Markov Decision Process (POMDP) framework for a partially observable and stochastically deteriorating syste...
متن کاملPlanning with Partially Observable Markov Decision Processes: Advances in Exact Solution Method
There is much interest in using par tially observable Markov decision processes (POMDPs) as a formal model for planning in stochastic domains. This paper is concerned with finding optimal policies for POMDPs. We propose several improvements to incre mental pruning, presently the most efficient exact algorithm for solving POMDPs.
متن کاملA Method for Speeding Up Value Iteration in Partially Observable Markov Decision Processes
We present a technique for speeding up the convergence of value iteration for par tially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov decision processes (MDPs). The technique can be easily incor porated into any existing POMDP value it eration algorithms. Experiments have been conducted on ...
متن کاملRegion-Based Incremental Pruning for POMDPs
We present a major improvement to the incremental pruning algorithm for solving partially observable Markov decision processes. Our technique targets the cross-sum step of the dynamic programming (DP) update, a key source of complexity in POMDP algorithms. Instead of reasoning about the whole belief space when pruning the cross-sums, our algorithm divides the belief space into smaller regions a...
متن کامل